Search for: All records

We study the convergence of several natural policy gradient (NPG) methods in infinite-horizon discounted Markov decision processes with regular policy parametrizations. For a variety of NPGs and reward functions we show that the trajectories in state-action space are solutions of gradient flows with respect to Hessian geometries, based on which we obtain global convergence guarantees and convergence rates. In particular, we show linear convergence for unregularized and regularized NPG flows with the metrics proposed by Kakade and Morimura and co-authors by observing that these arise from the Hessian geometries of conditional entropy and entropy respectively. Further, we obtain sublinear convergence rates for Hessian geometries arising from other convex functions like log-barriers. Finally, we interpret the discrete-time NPG methods with regularized rewards as inexact Newton methods if the NPG is defined with respect to the Hessian geometry of the regularizer. This yields local quadratic convergence rates of these methods for step size equal to the inverse penalization strength.

Mammalian megafauna have been critical to the functioning of Earth’s biosphere for millions of years. However, since the Plio-Pleistocene, their biodiversity has declined concurrently with dramatic environmental change and hominin evolution. While these biodiversity declines are well-documented, their implications for the ecological function of megafaunal communities remain uncertain. Here, we adapt ecometric methods to evaluate whether the functional link between communities of herbivorous, eastern African megafauna and their environments (i.e., functional trait-environment relationships) was disrupted as biodiversity losses occurred over the past 7.4 Ma. Herbivore taxonomic and functional diversity began to decline during the Pliocene as open grassland habitats emerged, persisted, and expanded. In the mid-Pleistocene, grassland expansion intensified, and climates became more variable and arid. It was then that phylogenetic diversity declined, and the trait-environment relationships of herbivore communities shifted significantly. Our results divulge the varying implications of different losses in megafaunal biodiversity. Only the losses that occurred since the mid-Pleistocene were coincident with a disturbance to community ecological function. Prior diversity losses, conversely, occurred as the megafaunal species and trait pool narrowed towards those adapted to grassland environments.

Understanding the relationships between functional traits and environment is increasingly important for assessing ecosystem health and forecasting biotic responses to future environmental change. Taxon-free analyses of functional traits (ecometrics) allow for testing the performance of such traits through time, utilizing both the fossil record and paleoenvironmental proxies. Here, we test the role of body size as a functional trait with respect to climate, using turtles as a model system. We examine the influence of mass-specific metabolic rate as a functional factor in the sorting of body size with environmental temperature and investigate the utility of community body size composition as an ecometric correlated to climate variables. We then apply our results to the fossil record of the Plio-Pleistocene Shungura Formation in Ethiopia. Results show that turtle body sizes scale with mass-specific metabolic rate for larger taxa, but not for the majority of species, indicating that metabolism is not a primary driver of size. Body size ecometrics have stronger predictive power at continental than at global scales, but without a single, dominant predictive functional relationship. Application of ecometrics to the Shungura fossil record suggests that turtle paleocommunity ecometrics coarsely track independent paleoclimate estimates at local scales. We hypothesize that both human disruption and biotic interactions limit the ecometric fit of size to climate in this clade. Nonetheless, examination of the consistency of trait–environment relationships through deep and shallow time provides a means for testing anthropogenic influences on ecosystems.